Contextual Dependency Vision Transformer for spectrogram-based multivariate time series analysis

Abstract

Multivariate time series (MTS) analysis plays an important role in various real-world applications. Existing Transformer-based methods address this problem based on hierarchical semantic representations across different scales. However, most of them ignore exploiting the helpful multiple temporal and variable relationships within the hierarchical semantic representations.

To this end, this paper proposes a novel method named Contextual Dependency Vision Transformer (CD-ViT), which generates multi-grained semantic information based on spectrogram and explores mutual dependencies between multi-variable and multi-temporal representations.

Key Modules

CD-ViT contains two key modules:

1. Hierarchical Variable-dependency Transformer (HVT) module: The HVT module progressively establishes mutual dependencies between multiple variables, from fine to coarse scales, with shared parameters.

2. Bidirectional Temporal-dependency Interaction (BTI) module: The BTI module employs two bidirectional flows to fuse multi-temporal tokens through zoom-in and zoom-out operations.

Experimental Results

Comprehensive experiments on widely used datasets, including UEA, Olszewski, UCI, MIMIC III, and ETT, demonstrate that the proposed approach achieves significant improvement on three popular tasks:

• Classification
• Regression
• Forecasting

The proposed CD-ViT method successfully captures contextual dependencies in both temporal and variable dimensions, leading to superior performance across multiple benchmarks and tasks.